A Framework for Representing and Reasoning about Three-Dimensional Objects for Vision

defined, the reasoning portion of the 3-D FORM system can apply the model to real data, recognizing instances of the defined objects and hypothesizing their missing parts. Figure 1 shows a portion of the knowledge that might be represented about a building. A BUILDING is a 3D-OBJECT; and a WALL, which is a 2D-OBJECT, is one of the building's parts. The specific building, BLDG1, has parts WALL1 and WALL2, whose geometric features are the primitive geometric objects PLANE1 and PLANE2, respectively. In addition to its geometric feature, WALL1 has the photometric feature COLOR1. The knowledge that a building's walls are mutually perpendicular is represented for BLDG1 by the instance PRPW12 of the PERPENDICULAR-PLANES relationship. The arguments to PRPW12 are PLANE1 and PLANE2, the geometric features of BLDG1's walls. The geometric feature PLANE1 is supported by the data object REGION1, which came from an image whose projection between 2-D and 3-D is CAM1. In the current implementation, only 3-D objects, geometric features, and geometric relationships are represented. The 3-D FORM system was applied to model-based interpretation of 3-D data and to solutions of sets of geometric constraints. This article discusses some of the issues in geometric reasoning for knowledge-based vision systems and how some existing systems have addressed them. Next, it describes the representations of primitive geometric objects, geometric relationships, and composite objects in the 3-D FORM system. Finally, it presents examples of the system's application to geometric reasoning tasks. e are developing the Framebased Object Recognition and Modeling (3-D FORM) System, a framework for representing and reasoning about three-dimensional (3-D) objects. This framework incorporates representations of objects, representations of the relationships between them, and a geometric reasoning capability. Such a representation and reasoning capability is essential for knowledge-based, 3-D photointerpretation systems that combine domain knowledge with image processing, as demonstrated by 3-D Mosaic (Herman, Kanade, and Kuroe 1984; Herman and Kanade 1986) and ACRONYM (Brooks 1981). It is also required for other applications, such as robot navigation, 3-D change detection, and the simulation of a scene's appearance from arbitrary viewpoints. The 3-D FORM system uses frames to represent objects such as buildings and walls, geometric features such as lines and planes, and geometric relationships such as parallel lines. Active procedures attached to the frames dynamically compute values as needed. Because processing is controlled by slot access, the system can hypothesize new objects or compute and verify relationships between existing objects, depending on the currently available knowledge. The 3-D FORM system can include knowledge about model and data objects organized into IS-A and PART hierarchies, along with relationships between object features, and projections used to convert between model objects and data objects. The knowledge includes both generic object models and specific instances of objects. Once the generic objects and the relationships between them are The capabilities for representing and reasoning about three-dimensional (3-D) objects are essential for knowledgebased, 3-D photointerpretation systems that combine domain knowledge with image processing, as demonstrated by 3D Mosaic and ACRONYM. Three-dimensional representation of objects is necessary for many additional applications, such as robot navigation and 3-D change detection. Geometric reasoning is especially important because geometric relationships between object parts are a rich source of domain knowledge. A practical framework for geometric representation and reasoning must incorporate projections between a two-dimensional (2-D) image and a 3-D scene, shape and surface properties of objects, and geometric and topological relationships between objects. In addition, it should allow easy modification and extension of the system's domain knowledge and be flexible enough to organize its reasoning efficiently to take advantage of the current available knowledge. We are developing such a framework—the Frame-based Object Recognition and Modeling (3-D FORM) System. This system uses frames to represent objects such as buildings and walls, geometric features such as lines and planes, and geometric relationships such as parallel lines. Active procedures attached to the frames dynamically compute values as needed. Because the order of processing is controlled largely by the order of slot access, the system performs both top-down and bottom-up reasoning, depending on the current available knowledge. The FORM system is being implemented with the Carnegie-Mellon University-built Framekit tool in Common Lisp (Carbonell and Joseph 1986). To date, it has been applied to two types of geometric reasoning problems: interpreting 3-D wire frame data and solving sets